Experimental support for layered images #719
Conversation
oracle-contributor-agreement
bot
added
the
OCA Verified
melix
changed the title
| .inheritIO() | ||
| .command("build/native/nativeCompile/layered-mn-app${IS_WINDOWS?".exe":""}") | ||
| def env = builder.environment() | ||
| env["LD_LIBRARY_PATH"] = testDirectory.resolve("build/native/nativeLibdependenciesCompile").toString() |
There was a problem hiding this comment.
Instead of setting LD_LIBRARY_PATH you can set -H:LinkerRPath={layer_path} when building the app layer to point to the directory containing the shared object(s) of underlying layer(s). The value gets passed to the linker as the -rpath. The path is relative to the directory where the executable resides. For example -H:LinkerRPath=$ORIGIN/lib points to the lib directory next to the executable.
There was a problem hiding this comment.
I had tried this without success. That said I'm not sure it's a good idea: this is only used for running the app, it is not required when building. So if we change this to use -H:LinkerRPath, wouldn't that mean that the binary wouldn't be portable anymore if the base layer paths change?
There was a problem hiding this comment.
That's correct, if you set -rpath then that gets hardcoded in the binary, but it's common for apps to depend on dirs relative to the main binary. I remember you had issues with LD_LIBRARY_PATH before, that's why I mentioned it. Also, you need to always be sure to not overwrite LD_LIBRARY_PATH in case other apps depend on it.
| main { | ||
| useLayer("libdependencies") | ||
| } | ||
| } |
There was a problem hiding this comment.
Can you please add an alternative layered configuration where only the java.base is in the base layer, e.g., something like:
libjavabase {
createLayer {
modules = ["java.base"]
}
}
main {
useLayer("libjavabase")
}
That would be useful for testing when something goes wrong by adding all the dependencies in the base layer. Similarly for the Micronaut test.
There was a problem hiding this comment.
Not sure what you mean, since the DSL already allows that. Or do you want to have an additional test? We try not to add too many redundant tests, since the build is already taking ages.
| main { | ||
| useLayer("libdependencies") | ||
| } | ||
| } |
There was a problem hiding this comment.
Can you please also add a configuration that could switch off the layered build? In your early demo if I recall correctly you had an -incremental flag. It would be useful to be able to easily switch between these 3 scenarios for demo purposes (and maybe others in the future):
- a regular build (no layers)
- a layered build that has just
java.basein the base layer - a layered build that has
java.baseand all dependencies in the base layer
There was a problem hiding this comment.
Again it's unclear to me what you want. These are tests of the plugin itself, it doesn't quite make sense to add these options, since they wouldn't be used by our tests.
There was a problem hiding this comment.
I see, then what would be the best way to showcase this plugin functionality to an end user?
There was a problem hiding this comment.
Maybe after we merge the plugin changes we can add a demo project to the Micronaut repo?
There was a problem hiding this comment.
Yes. Although once this is merged and NBT released, I could implement this directly in the Micronaut Gradle plugins, where the options would make more sense, and the configuration could be hidden.
| @@ -45,7 +45,7 @@ import org.graalvm.buildtools.gradle.fixtures.AbstractFunctionalTest | |||
|
|
|||
| class JUnitFunctionalTests extends AbstractFunctionalTest { | |||
| def "test if JUint support works with various annotations, reflection and resources"() { | |||
|
|
|||
| debug=true | |||
native-gradle-plugin/src/main/java/org/graalvm/buildtools/gradle/NativeImagePlugin.java
Outdated
Show resolved
Hide resolved
...gin/src/main/java/org/graalvm/buildtools/gradle/internal/NativeImageCommandLineProvider.java
Outdated
Show resolved
Hide resolved
...gin/src/main/java/org/graalvm/buildtools/gradle/internal/NativeImageCommandLineProvider.java
Outdated
Show resolved
Hide resolved
This commit introduces a Gradle DSL to support layered images creation.
As of now, this is mainly aimed towards a single use case which is incremental builds.
The test project demonstrates how to use the DSL to create a base layer which includes
the JDK "java.base" module as well as all external dependencies used by the project.
The DSL builds on top of the binaries concept, by allowing them to declare either
that they produce a layer, or that they use one or more layers. The DSL contains
methods to make it easier to declare use or creation, as well as supports an easy
way to declare that a layer should use only external dependencies. For example,
this is how you would create a base layer and consume it from the main binary:
```
graalvmNative {
binaries {
libdependencies {
createLayer {
modules = ["java.base"]
jars.from(externalDependenciesOf(configurations.runtimeClasspath))
}
}
main {
useLayer("libdependencies")
}
}
}
```
Note that there is a _binary_ named `libdependencies`, and as soon
as the `createLayer` is called, it will offer additional options which are
specific to layers (for example declaring the list of packages or modules).
The DSL to create a layer contains the `packages` option, which could be
used with automatic extraction of package names, which is why there is
code to extract packages from jars, however, this code is currently
unused for a reason: these packages can contain dependencies to "optional"
modules, which cannot be figured out at build time. Typically, logback
will support additional modules and load them dynamically, and if the
package is included in the list and that the supporting dependencies
are not on classpath, then the layer creation would fail. Therefore,
the only reliable option right now is to use the `jars` property to
set the list of jars which should belong to the layer.
Windows uses the argFile by default, which highlighted a bug: the arg file was written in the output directory, which was cleaned before running the task (cleaning is necessary or we'd have stale files and up-to-date checking would be broken). The arg file is now written in the temp directory instead.
This commit introduces a Gradle DSL to support layered images creation. As of now, this is mainly aimed towards a single use case which is incremental builds. The test project demonstrates how to use the DSL to create a base layer which includes the JDK "java.base" module as well as all external dependencies used by the project.
The DSL builds on top of the binaries concept, by allowing them to declare either that they produce a layer, or that they use one or more layers. The DSL contains methods to make it easier to declare use or creation, as well as supports an easy way to declare that a layer should use only external dependencies. For example, this is how you would create a base layer and consume it from the main binary:
Note that there is a binary named
libdependencies, and as soon as thecreateLayeris called, it will offer additional options which are specific to layers (for example declaring the list of packages or modules).The DSL to create a layer contains the
packagesoption, which could be used with automatic extraction of package names, which is why there is code to extract packages from jars, however, this code is currently unused for a reason: these packages can contain dependencies to "optional" modules, which cannot be figured out at build time. Typically, logback will support additional modules and load them dynamically, and if the package is included in the list and that the supporting dependencies are not on classpath, then the layer creation would fail. Therefore, the only reliable option right now is to use thejarsproperty to set the list of jars which should belong to the layer.